Selective color display of a thermal image

ABSTRACT

An imaging system includes an array of photodetectors that produce an array of intensity values. Image and display processing components of the imaging system produce an array of display-formatted pixels for display on an imaging system display. The display-formatted pixels include a first plurality of pixels associated with a color lookup table and a second plurality of pixels associated with a grey-scale lookup table. The image and display processing components compare pixel intensity values to temperature criteria and apply the color lookup table to pixels satisfying the temperature criteria and apply the grey-scale lookup table to pixels not satisfying the temperature criteria.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority from U.S. Prov. App'nNo. 62/049,880, filed Sep. 12, 2014, entitled “Selective Color Displayof a Thermal Image,” which is incorporated by reference herein in itsentirety.

BACKGROUND

Field

The present disclosure generally relates to color display of a thermalimage, and in particular to using different display lookup tablesdepending on temperatures in a scene.

Description of Related Art

The increasing availability of high-performance, low-cost uncooledinfrared imaging devices, such as bolometer focal plane arrays (FPAs),is enabling the design and production of mass-produced,consumer-oriented infrared (IR) cameras capable of quality thermalimaging. Such thermal imaging sensors have long been expensive anddifficult to produce, thus limiting the employment of high-performance,long-wave imaging to high-value instruments, such as aerospace,military, or large-scale commercial applications. Mass-produced IRcameras may have different design requirements than complex military orindustrial systems. New approaches to provide effective thermal imagedisplay with limited camera resources may be beneficial.

SUMMARY

Example embodiments described herein have innovative features, no singleone of which is indispensable or solely responsible for their desirableattributes. Without limiting the scope of the claims, some of theadvantageous features will now be summarized.

An imaging system includes an array of photodetectors configured toproduce an array of intensity values corresponding to light intensity atthe photodetectors. The imaging system can include a display for displayimages acquired with the array of photodetectors, after some image anddisplay processing. The image and display processing components of theimaging system produce an array of display-formatted pixels for displayon the imaging system display. The display-formatted pixels include afirst plurality of pixels formatted for display using a first lookuptable and a second plurality of pixels formatted for display using asecond lookup table. The first lookup table can be a color lookup tableand the second lookup table can be a grey scale lookup table, forexample and without limitation. To determine which lookup table to use,the image and display processing components compare pixel values totemperature criteria and format pixels using the first lookup table forpixels satisfying the temperature criteria and format pixels using thesecond lookup table for pixels not satisfying the temperature criteria.The temperature criteria can be defined by a user of the imaging system,for example and without limitation. The imaging system can be a thermalimaging system and may include an infrared camera core.

In a first aspect, a method is provided for displaying a thermal imageusing a thermal imaging system having an array of photodetectorsconfigured to acquire thermal image data. The method includes acquiringthermal image data with the array of photodetectors, the thermal imagedata comprising an array of pixel intensity values. The method includesreceiving temperature criteria, determining a scene temperature forindividual pixel intensity values using a thermography function, andcomparing individual pixels to the temperature criteria. The method alsoincludes formatting for display individual pixel values. The method alsoincludes applying a first lookup table to an individualdisplay-formatted pixel if the pixel value satisfies the temperaturecriteria and applying a second lookup table to the individualdisplay-formatted pixel if the pixel value does not satisfy thetemperature criteria. The method also includes displaying a thermalimage on a display of the thermal imaging system, the thermal imagecomprising the display-formatted individual pixel values with theapplied lookup tables.

In some embodiments of the first aspect, the method further includesconverting the temperature criteria to intensity-based criteria using athermography function. In a further embodiment, comparing individualpixels to the temperature criteria includes comparing individual pixelintensity values to the intensity-based criteria. In some embodiments ofthe first aspect, comparing individual pixels to the temperaturecriteria includes comparing individual pixel temperature values to thetemperature criteria. In some embodiments of the first aspect, the firstlookup table is configured to output a color value and the second lookuptable is configured to output a grey-scale value, and wherein at least aportion of the thermal image is displayed using color and anotherportion of the thermal image is displayed using a grey scale.

In some embodiments of the first aspect, the temperature criteriacomprises at least one of a minimum temperature threshold that issatisfied if an individual pixel intensity value corresponds to atemperature that is greater than or equal to the minimum temperaturethreshold; a maximum temperature threshold that is satisfied if anindividual pixel intensity value corresponds to a temperature that isless than or equal to the maximum temperature threshold; a targetedtemperature value that is satisfied if an individual pixel intensityvalue corresponds to a temperature that is substantially the same as thetargeted temperature value; and a targeted temperature range that issatisfied if an individual pixel intensity value corresponds to atemperature that is within the targeted temperature range.

In some embodiments of the first aspect, receiving the temperaturecriteria includes determining the temperature criteria based on userinteraction with the thermal imaging system. In some embodiments of thefirst aspect, the temperature criteria includes a plurality ofconditions. In a further embodiment, the method further includesapplying a first color lookup table if the individual pixel valuesatisfies a first condition of the temperature criteria, applying asecond color lookup table if the individual pixel value satisfies asecond condition of the temperature criteria, and applying a grey-scalelookup table if the individual pixel value does not satisfy any of theplurality of conditions.

In some embodiments of the first aspect, the method further includesdisplaying at least one scene temperature overlaid on the thermal image.In some embodiments of the first aspect, the method further includesperforming signal processing on the thermal image data, the signalprocessing including performing a histogram equalization method on thepixel intensity values prior to applying the first or second lookuptables. In some embodiments of the first aspect, the method furtherincludes displaying a user interface with which a user may interact toset the temperature criteria.

In a second aspect, a thermal imaging system is provided that includesan imaging array comprising an infrared focal plane array, the infraredfocal plane array configured to generate signals corresponding to levelsof infrared light incident on the infrared focal plane array. Thethermal imaging system includes a detector circuit comprising readoutelectronics that receive the generated signals and output an array ofpixel intensity values. The thermal imaging system includes a systemcontroller configured to receive temperature criteria, determine a scenetemperature for individual pixel intensity values using a thermographyfunction, and compare individual pixels to the temperature criteria. Thesystem controller is further configured to format for display individualpixel intensity values. The system controller is further configured toapply a first lookup table to an individual display-formatted pixel ifthe individual pixel intensity value satisfies the intensity-basedcriteria and apply a second lookup table to the individualdisplay-formatted pixel if the individual pixel intensity value does notsatisfy the intensity-based criteria. The thermal imaging systemincludes a display configured to display a thermal image comprising anarray of the display-formatted pixels with the applied lookup tables.

In some embodiments of the second aspect, the system controller isfurther configured to convert the temperature criteria tointensity-based criteria using the thermography function. In a furtherembodiment, the system controller is configured to compare individualpixels to the temperature criteria by comparing individual pixelintensity values to the intensity-based criteria.

In some embodiments of the second aspect, the first lookup tablecomprises a color lookup table and the second lookup table comprises agrey-scale lookup table such that the thermal image comprises at least afirst plurality of pixels displayed using color and a second pluralityof pixels displayed using grey scale. In some embodiments of the secondaspect, the display comprises a touchscreen interface and thetemperature criteria is received through user interaction with thetouchscreen interface.

In some embodiments of the second aspect, the thermal imaging systemincludes a thermal camera comprising a thermal camera housing and aconnector wherein the imaging array and the detector circuit are withinthe thermal camera housing, and a personal electronics device comprisingthe system controller and the display, wherein the connector of thethermal camera is configured to mechanically and electrically couple tothe personal electronics device to transmit the thermal image data fromthe thermal camera to the personal electronics device. In a furtherembodiment, control of the thermal camera is provided throughinteraction with the personal electronics device.

In some embodiments of the second aspect, the temperature criteriacomprises at least one of a minimum temperature threshold that issatisfied if an individual pixel intensity value corresponds to atemperature that is greater than or equal to the minimum temperaturethreshold; a maximum temperature threshold that is satisfied if anindividual pixel intensity value corresponds to a temperature that isless than or equal to the maximum temperature threshold; a targetedtemperature value that is satisfied if an individual pixel intensityvalue corresponds to a temperature that is substantially the same as thetargeted temperature value; and a targeted temperature range that issatisfied if an individual pixel intensity value corresponds to atemperature that is within the targeted temperature range.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects and advantages of the embodiments provided herein are describedwith reference to the following detailed description in conjunction withthe accompanying drawings. Throughout the drawings, reference numbersmay be re-used to indicate correspondence between referenced elements.The drawings are provided to illustrate example embodiments describedherein and are not intended to limit the scope of the disclosure.

FIG. 1A illustrates a functional block diagram of an example imagingsystem.

FIG. 1B illustrates a functional block diagram of the example imagingsystem illustrated in FIG. 1A, wherein functionality of the imagingsystem is divided between a camera and a mobile electronic device.

FIGS. 2A and 2B illustrate an example thermal imaging camera and theexample thermal imaging camera interfaced to a personal electronicsdevice.

FIG. 3 illustrates a functional block diagram of a thermal imagingsystem configured to convert thermal image data acquired with a thermalimage sensor to an array of display-formatted pixels, thedisplay-formatted pixels being formatted using at least two differentlookup tables depending on display formatting criteria.

FIGS. 4A and 4B illustrate an example color display of a thermal image,the color portions of the displayed image corresponding to temperaturesabove a temperature threshold.

FIGS. 5A and 5B illustrate an example color display of a thermal image,the color portions of the displayed image corresponding to temperaturesequal to a temperature value.

FIG. 6 illustrates an example method of selective color display for athermal imaging system.

DETAILED DESCRIPTION

Generally described, aspects of the present disclosure relate toselective color display of a thermal image on a thermal imaging system.The selective color display can be configured to depend at least in parton pixel intensity values and how those intensity values translate intoscene temperature. Pixels satisfying a temperature criteria can bedisplayed using colors and pixels not satisfying the temperaturecriteria can be displayed using a grey scale. The present disclosureincludes systems and methods to format pixels for display using one ormore lookup tables to translate pixel intensity values to pixel displayvalues. To determine these pixel display values, the systems and methodsdisclosed herein compare the pixel intensity values to temperaturecriteria, select a lookup table based on satisfaction ornon-satisfaction of the temperature criteria, and retrieve a pixeldisplay value corresponding to the pixel intensity value from theselected lookup table. These pixels can then be displayed on the thermalimaging system. Thus, in some embodiments, these systems and methods canselectively display portions of an image using a range of colors andother portions of the image using grey scale. Advantageously, this canallow a user to quickly determine which portions of an image satisfy thecriteria. Moreover, this can allow the thermal imaging system toefficiently process and format pixels for display using a plurality oflookup tables for the display formatting.

Although examples and implementations described herein focus, for thepurpose of illustration, on implementation in an infrared camera and forthermal images, the systems and methods disclosed herein can beimplemented in digital and/or video cameras that acquire visible lightusing a variety of image sensors. Various aspects of the disclosure willnow be described with regard to certain examples and embodiments, whichare intended to illustrate but not limit the disclosure.

Thermal imaging and display systems can be configured to receive or todetermine a temperature value of interest (e.g., such as from a user ofthe system). When a thermal image of a scene is displayed, the varioustemperatures of the scene may be displayed by formatting pixel valuesusing different lookup tables depending on the relationship of pixelintensity values to the temperature value of interest. In someembodiments, temperatures in a scene satisfying a tailored criteria(e.g., temperatures within a tailored range, above a threshold, below athreshold, equal to a temperature value, etc.) may be displayed using acolor lookup table, and the other temperatures in the scene may bedisplayed using a grey-scale lookup table.

Thermal imaging systems may implement a method for displaying thermalimages that selectively colorize portions of the thermal image based onsatisfaction of temperature criteria. The method can include acquiringimage data of a scene from a thermal imaging sensor comprising an arrayof photodetectors (e.g., pixels), wherein the image data corresponds tothe intensity of radiation detected by the pixels of the imaging sensor.The method can then be configured to determine the relation betweendetected intensity and scene temperature with a thermography function.The method can be configured to perform signal processing on the imagedata, which may include formatting the image data for presentation on avisual display. The method can receive or determine a temperature valueof interest (e.g., such as from a user) and can convert the temperaturevalue of interest to a corresponding intensity value of interest withthe thermography function. The method can compare pixel intensity valuesfor individual pixels to the intensity value of interest. The method canselect a display lookup table from a plurality of available lookuptables based at least in part on the relationship between the pixelintensity and the intensity value of interest and format for display thepixel data with the selected lookup table for each pixel displayed. Insome embodiments, at least one of the selected lookup tables can be acolor lookup table and at least one other selected lookup table can be agrey-scale lookup table.

Some embodiments described herein advantageously improve thepresentation of thermal images by increasing the amount of visualinformation available in the displayed image by selectively colorizingat least a portion of the thermal image based on satisfaction oftemperature criteria. A user can thus quickly ascertain which portionsof a thermal image satisfy the temperature criteria by identifying whichportions are displayed in color.

Some embodiments described herein advantageously provide for display ofa thermal image with regions of interest highlighted for display. Theuser can provide temperature criteria to a thermal imaging system andregions of the thermal image that satisfy the temperature criteria canbe highlighted (e.g., by using colors rather than grey scale).

Some embodiments described herein advantageously provide an efficientway to improve the user experience on a mass-produced thermal imagingcamera. Through the use of a plurality of lookup tables and user-inputtemperature criteria, the thermal imaging camera can efficientlydetermine and colorize portions of a thermal image to provide a visualindication of satisfaction of the user-input criteria. This feature mayenhance the user experience by providing a visually distinctiveindication of regions of interest to the user.

The disclosed systems and methods for displaying a thermal image may beimplemented as modules that may be a programmed computer method or adigital logic method and may be implemented using a combination of anyof a variety of analog and/or digital discrete circuit components(transistors, resistors, capacitors, inductors, diodes, etc.),programmable logic, microprocessors, microcontrollers,application-specific integrated circuits, or other circuit elements. Amemory configured to store computer programs or computer-executableinstructions may be implemented along with discrete circuit componentsto carry out one or more of the methods described herein. In certainimplementations, the disclosed methods may be implemented in conjunctionwith a focal plane array (FPA) on a camera core, wherein the processorand memory components executing the disclosed methods may be on a devicemated to the camera core, such as a mobile appliance including smartphones, tablets, personal computers, etc. In some implementations, theprocessing and memory elements of the imaging system may be inprogrammable logic or on-board processors that are part of the core orcamera system. In some embodiments, image gain calibration may beaccomplished on a processing element on the camera core, and furtherimage processing and display may be accomplished by a system controllermated to the core.

As a particular example of some advantages provided by the disclosedsystems and methods, an imaging system can include a focal plane array(FPA) configured to acquire images of a scene. The FPA can include atwo-dimensional array of N detectors, the FPA configured to output atwo-dimensional image of the scene. For imaging purposes, image frames,typically data from all or some of the detectors N_(f), are produced bythe FPA, each successive frame containing data from the array capturedin successive time windows. Thus, a frame of data delivered by the FPAcomprises N_(f) digital words, each word representing a particularpixel, P, in the image. These digital words are usually of a lengthdetermined by the analog to digital conversion (A/D) process. Forexample, if the pixel data is converted with a 14 bit A/D, the pixelwords may be 14 bits in length, and there may be 16384 counts per word.For an IR camera used as a thermal imaging system, these words maycorrespond to an intensity of radiation measured by each pixel in thearray. In a particular example, for a bolometer IR FPA the intensity perpixel usually corresponds to temperature of the corresponding part ofthe imaged scene, with lower values corresponding to colder regions andhigher values to hotter regions. It may be desirable to display thisdata on a visual display.

Example Imaging Systems

FIG. 1A illustrates a functional block diagram of an imaging system 100comprising an image sensor such as a focal plane array 102, apre-processing module 104, a non-uniformity correction module 106, afilter module 108, a thermography module 110, a histogram equalizationmodule 112, a display processing module 114, and a display 116. Thefocal plane array 102 can output a sequence of frames of intensity data(e.g., images, thermal images, etc.). Each frame can include an array ofpixel values, each pixel value representing light intensity detected bya corresponding pixel on the focal plane array 102. The pixel values canbe read out of the focal plane array 102 as a stream of serial digitaldata. In some embodiments, the pixel values are read out of the focalplane array 102 using read out electronics that process whole rows orwhole columns of the focal plane array 102. The format of the stream ofdata can be configured to conform to a desired, standard, or pre-definedformat. The stream of digital data can be displayed as a two-dimensionalimage, such as by the display 116.

In some embodiments, the focal plane array 102 can be an array ofmicrobolometers integrated with a read out integrated circuit (“ROIC”).The array of microbolometers can be configured to generate electricalsignals in response to a quantity of thermal radiation or a temperature.The ROIC can include buffers, integrators, analog-to-digital converters,timing components, and the like to read the electrical signals from thearray of microbolometers and to output a digital signal (e.g., 14-bitserial data separated into image frames). Additional examples of systemsand methods associated with the focal plane array 102 are disclosed inU.S. patent application Ser. No. 14/292,124, entitled “Data Digitizationand Display for an Imaging System,” filed May 30, 2014, the entirecontents of which is incorporated by reference herein.

The focal plane array 102 can have calibration or other monitoringinformation associated with it (e.g., calibration data 103) that can beused during image processing to generate a superior image. For example,calibration data 103 may include bad pixel maps and/or gain tablesstored in data storage and retrieved by modules in the imaging system100 to correct and/or adjust the pixel values provided by the focalplane array 102. Calibration data 103 may include gain tables. Asdescribed herein, the focal plane array 102 can include a plurality ofpixels with integrated read out electronics. The read out electronicscan have a gain associated with it, wherein the gain may be proportionalto the transimpedance of a capacitor in the electronics. This gainvalue, which may in some implementations take the form of a pixel gaintable, may be used by the image processing modules of the imaging system100. Additional examples of calibration data for the imaging system 100are described in greater detail in U.S. patent application Ser. No.14/829,490, entitled “Gain Calibration for an Imaging System,” filedAug. 18, 2015, the entire contents of which is incorporated by referenceherein. The calibration data 103 can be stored on the imaging system 100or in data storage on another system for retrieval during imageprocessing.

The imaging system 100 includes one or more modules configured toprocess image data from the focal plane array 102. One or more of themodules of the imaging system 100 can be eliminated without departingfrom the scope of the disclosed embodiments. The following modules aredescribed to illustrate the breadth of functionality available to thedisclosed imaging systems and not to indicate that any individual moduleor described functionality is required, critical, essential, ornecessary.

The imaging system 100 includes the pre-processing module 104. Thepre-processing module 104 can be configured to receive the digital datastream from the focal plane array 102 and to perform pre-processingfunctions. Examples of such functions include frame averaging,high-level frame-wide filtering, etc. The pre-processing module 104 canoutput serial digital data for other modules.

As an example, the pre-processing module 104 can include conditionalsummation functionality configured to implement integration andaveraging techniques to increase apparent signal to noise in image data.For example, the conditional summation functionality can be configuredto combine successive frames of digitized image data to form a digitallyintegrated image. This digitally integrated image can also be averagedto reduce noise in the image data. The conditional summationfunctionality can be configured to sum values from successive frames foreach pixel from the focal plane array 102. For example, the conditionalsummation functionality can sum the values of each pixel from foursuccessive frames and then average that value. In some implementations,the conditional summation functionality can be configured to select abest or preferred frame from successive frames rather than summing thesuccessive frames. Examples of these techniques and additionalembodiments are disclosed in U.S. patent application Ser. No.14/292,124, entitled “Data Digitization and Display for an ImagingSystem,” filed May 30, 2014, the entire contents of which isincorporated by reference herein.

As another example, the pre-processing module 104 can include adaptiveresistor digital to analog converter (“RDAC”) functionality configuredto determine and/or adjust for operating bias points of the focal planearray 102. For example, for an imaging system that includes a shutter,the imaging system 100 can be configured to adjust an operating biaspoint of the detectors in the focal plane array 102. The adaptive RDACfunctionality can implement an adaptive operating bias correction methodthat is based at least in part on periodic measurement of a flat fieldimage (e.g., an image acquired with the shutter closed). The adaptiveRDAC functionality can implement an ongoing adjustment of the operatingbias based at least in part on a measured or detected drift over time ofthe flat field image. The bias adjustment provided by the adaptive RDACfunctionality may provide compensation for drift over time of thephotodetectors and electronics due to effects such as temperaturechanges. In some embodiments, the adaptive RDAC functionality includesan RDAC network that can be adjusted to bring measured flat field datacloser to a reference bias level. Additional examples of systems andmethods related to the adaptive RDAC functionality are described ingreater detail in U.S. patent application Ser. No. 14/829,500, filedAug. 18, 2015, entitled “Adaptive Adjustment of the Operating Bias of anImaging System,” the entire contents of which is incorporated byreference herein.

After the pre-processing module 104, other processing modules can beconfigured to perform a series of pixel-by-pixel or pixel groupprocessing steps. For example, the image processing system 100 includesa non-uniformity correction module 106 configured to adjust pixel datafor gain and offset effects that are not part of the image scene itself,but are artifacts of the sensor. For example, the non-uniformitycorrection module 106 can be configured to receive a stream of digitaldata and correct pixel values for non-uniformities in the focal planearray 102. In some imaging systems, these corrections may be derived byintermittently closing a shutter over the focal plane array 102 toacquire uniform scene data. From this acquired uniform scene data, thenon-uniformity correction module 106 can be configured to determinedeviations from uniformity. The non-uniformity correction module 106 canbe configured to adjust pixel data based on these determined deviations.In some imaging systems, the non-uniformity correction module 106utilizes other techniques to determine deviations from uniformity in thefocal plane array. Some of these techniques can be implemented withoutthe use of a shutter. Additional examples of systems and methods fornon-uniformity correction are described in U.S. patent application Ser.No. 14/817,847, entitled “Time Based Offset Correction for ImagingSystems,” filed Aug. 4, 2015, the entire contents of which isincorporated by reference herein.

After the pre-processing module 104, the imaging system 100 can includea high/low C_(int) signal processing functionality configured to receivea stream of digital data (e.g., 14-bit serial data) from thepre-processing module 104. The high/low C_(int) functionality can beconfigured to process the stream of digital data by applying gaintables, for example, as provided in the calibration data 103. Thehigh/low C_(int) functionality can be configured to process the streamof digital data using output of high/low integration components. Suchhigh/low integration components can be integrated with the ROICassociated with the focal plane array 102. Examples of the high/lowintegration components are described in U.S. patent application Ser. No.14/292,124, entitled “Data Digitization and Display for an ImagingSystem,” filed May 30, 2014, the entire contents of which isincorporated by reference herein.

The image processing system 100 includes a filter module 108 configuredto apply one or more temporal and/or spatial filters to address otherimage quality issues. For example, the read out integrated circuit ofthe focal plane array can introduce artifacts into an image, such asvariations between rows and/or columns. The filter module 108 can beconfigured to correct for these row- or column-based artifacts, asdescribed in greater detail in U.S. patent application Ser. No.14/702,548, entitled “Compact Row Column Noise Filter for an ImagingSystem,” filed May 1, 2015, the entire contents of which is incorporatedby reference herein. The filter module 108 can be configured to performcorrections to reduce or eliminate effects of bad pixels in the image,enhance edges in the image data, suppress edges in the image data,adjust gradients, suppress peaks in the image data, and the like.

For example, the filter module 108 can include bad pixel functionalityconfigured to provide a map of pixels on the focal plane array 102 thatdo not generate reliable data. These pixels may be ignored or discarded.In some embodiments, data from bad pixels is discarded and replaced withdata derived from neighboring, adjacent, and/or near pixels. The deriveddata can be based on interpolation, smoothing, averaging, or the like.

As another example, the filter module 108 can include thermal gradientfunctionality configured to adjust pixel values based on thermalgradients present in the image data but that are not part of the sceneimaged by the imaging system 100. The thermal gradient functionality canbe configured to use local flat scene data to derive data to improveimage quality by correcting for thermal gradients produced in theimaging system 100. Examples of determining corrections for the thermalgradient functionality are described in greater detail in U.S. Prov.Pat. App'n No. 62/086,305, entitled “Image Adjustment Based on LocallyFlat Scenes,” filed Dec. 2, 2014, the entire contents of which isincorporated by reference herein.

The filter module 108 can include peak limit functionality configured toadjust outlier pixel values. For example, the peak limit functionalitycan be configured to clamp outlier pixel values to a threshold value.

The filter module 108 can be configured to include an adaptive low-passfilter and/or a high-pass filter. In some embodiments, the imagingsystem 100 applies either the adaptive low-pass filter or the high-passfilter, but not both. The adaptive low-pass filter can be configured todetermine locations within the pixel data where it is likely that thepixels are not part of an edge-type image component. In these locations,the adaptive low-pass filter can be configured to replace pixel datawith smoothed pixel data (e.g., replacing pixel values with the averageor median of neighbor pixels). This can effectively reduce noise in suchlocations in the image. The high-pass filter can be configured toenhance edges by producing an edge enhancement factor that may be usedto selectively boost or diminish pixel data for the purpose of edgeenhancement. Additional examples of adaptive low-pass filters andhigh-pass filters are described in U.S. patent application Ser. No.14/817,989, entitled “Local Contrast Adjustment for Digital Images,”filed Aug. 4, 2015, the entire contents of which is incorporated byreference herein.

The filter module 108 can be configured to apply optional filters to theimage data. For example, optional filters can include, withoutlimitation, averaging filters, median filters, smoothing filters, andthe like. The optional filters can be turned on or off to providetargeted or desired effects on the image data.

The image processing system 100 includes a thermography module 110configured to convert intensity to temperature. The light intensity cancorrespond to intensity of light from a scene and/or from objects in afield of view of the imaging system 100. The thermography module 110 canbe configured to convert the measured light intensities to temperaturescorresponding to the scene and/or objects in the field of view of theimaging system 100. The thermography module 110 can receive as inputcalibration data (e.g., calibration data 103). The thermography module110 may also use as inputs raw image data (e.g., pixel data from thepre-processing module 104) and/or filtered data (e.g., pixel data fromthe filter module 108). Examples of thermography modules and methods areprovided in U.S. patent application Ser. No. 14/838,000, entitled“Thermography for a Thermal Imaging Camera,” filed Aug. 27, 2015, theentire contents of which is incorporated by reference herein.

The image processing system 100 includes a histogram equalization module112, or other display conversion module, configured to prepare the imagedata for display on the display 116. In some imaging systems, thedigital resolution of the pixel values from the focal plane array 102can exceed the digital resolution of the display 116. The histogramequalization module 112 can be configured to adjust pixel values tomatch the high resolution value of an image or a portion of an image tothe lower resolution of the display 116. The histogram module 112 can beconfigured to adjust pixel values of the image in a manner that avoidsusing the limited display range of the display 116 on portions of theimage where there is little or no data. This may be advantageous for auser of the imaging system 100 when viewing images acquired with theimaging system 100 on the display 116 because it can reduce the amountof display range that is not utilized. For example, the display 116 mayhave a digital brightness scale, which for an infrared image correspondsto temperature where higher intensity indicates a higher temperature.However, the display brightness scale, for example a grey scale, isgenerally a much shorter digital word than the pixel sample words. Forinstance, the sample word of the pixel data may be 14 bits while adisplay range, such as grey scale, can be typically 8 bits. So fordisplay purposes, the histogram equalization module 112 can beconfigured to compress the higher resolution image data to fit thedisplay range of the display 116. Examples of algorithms and methodsthat may be implemented by the histogram equalization module 112 aredisclosed in U.S. patent application Ser. No. 14/292,124, entitled “DataDigitization and Display for an Imaging System,” filed May 30, 2014, theentire contents of which is incorporated by reference herein.

The imaging system 100 includes a display processing module 114configured to prepare the pixel data for display on the display 116 by,for example, selecting color tables to convert temperatures and/or pixelvalues to color on a color display. As an example, the displayprocessing module can include a colorizer lookup table configured toconvert pixel data and/or temperature data into color images for displayon the display 116. The colorizer lookup table can be configured todisplay different temperatures of a thermally imaged scene usingdifferent color display lookup tables depending at least in part on therelationship of a temperature of a given scene to a thresholdtemperature. For example, when a thermal image of a scene is displayed,various temperatures of the scene may be displayed using differentlookup tables depending on their relationship to the input temperature.In some embodiments, temperatures above, below, or equal to an inputtemperature value may be displayed using a color lookup table, whileother temperatures may be displayed using a grey scale lookup table.Accordingly, the colorizer lookup table can be configured to applydifferent colorizing lookup tables depending on temperature rangeswithin a scene in combination with user preferences or selections.Additional examples of functionality provided by a display processingmodule are described herein with reference to FIGS. 2A-6.

The display 116 can be configured display the processed image data. Thedisplay 116 can also be configured to accept input to interact with theimage data and/or to control the imaging system 100. For example, thedisplay 116 can be a touchscreen display.

The imaging system 100 can be provided as a standalone device, such as athermal sensor. For example, the imaging system 100 can include animaging system housing configured to enclose hardware components (e.g.,the focal plane array 102, read out electronics, microprocessors, datastorage, field programmable gate arrays and other electronic components,and the like) of the imaging system 100. The imaging system housing canbe configured to support optics configured to direct light (e.g.,infrared light, visible light, etc.) onto the image sensor 102. Thehousing can include one or more connectors to provide data connectionsfrom the imaging system 100 to one or more external systems. The housingcan include one or more user interface components to allow the user tointeract with and/or control the imaging system 100. The user interfacecomponents can include, for example and without limitation, touchscreens, buttons, toggles, switches, keyboards, and the like.

In some embodiments, the imaging system 100 can be part of a network ofa plurality of imaging systems. In such embodiments, the imaging systemscan be networked together to one or more controllers.

FIG. 1B illustrates a functional block diagram of the example imagingsystem 100 illustrated in FIG. 1A, wherein functionality of the imagingsystem 100 is divided between a camera or sensor 140 and a mobileelectronic device 150. By dividing image acquisition, pre-processing,signal processing, and display functions among different systems ordevices, the camera 140 can be configured to be relatively low-power,relatively compact, and relatively computationally efficient compared toan imaging system that performs a majority or all of such functions onboard. As illustrated in FIG. 1B, the camera 140 is configured toinclude the focal plane array 102 and the pre-processing module 104. Insome embodiments, one or more of the modules illustrated as being partof the mobile electronic device 150 can be included in the camera 140instead of in the mobile electronic device 150. In some embodiments,certain advantages are realized based at least in part on the divisionof functions between the camera 140 and the mobile electronic device150. For example, some pre-processing functions can be implementedefficiently on the camera 140 using a combination of specializedhardware (e.g., field-programmable gate arrays, application-specificintegrated circuits, etc.) and software that may otherwise be morecomputationally expensive or labor intensive to implement on the mobileelectronic device 150. Accordingly, an aspect of at least some of theembodiments disclosed herein includes the realization that certainadvantages may be achieved by selecting which functions are to beperformed on the camera 140 (e.g., in the pre-processing module 104) andwhich functions are to be performed on the mobile electronic device 150(e.g., in the thermography module 110).

An output of the camera 140 can be a stream of digital data representingpixel values provided by the pre-processing module 104. The data can betransmitted to the mobile electronic device 150 using electronicconnectors (e.g., a micro-USB connector, proprietary connector, etc.),cables (e.g., USB cables, Ethernet cables, coaxial cables, etc.), and/orwirelessly (e.g., using BLUETOOTH, Near-Field Communication, Wi-Fi,etc.). The mobile electronic device 150 can be a smartphone, tablet,laptop, or other similar portable electronic device. In someembodiments, power is delivered to the camera 140 from the mobileelectronic device 150 through the electrical connectors and/or cables.

The imaging system 100 can be configured to leverage the computingpower, data storage, and/or battery power of the mobile electronicdevice 150 to provide image processing capabilities, power, imagestorage, and the like for the camera 140. By off-loading these functionsfrom the camera 140 to the mobile electronic device 150, the camera canhave a cost-effective design. For example, the camera 140 can beconfigured to consume relatively little electronic power (e.g., reducingcosts associated with providing power), relatively little computationalpower (e.g., reducing costs associated with providing powerfulprocessors), and/or relatively little data storage (e.g., reducing costsassociated with providing digital storage on the camera 140). This canreduce costs associated with manufacturing the camera 140 due at leastin part to the camera 140 being configured to provide relatively littlecomputational power, data storage, and/or power, because the imagingsystem 100 leverages the superior capabilities of the mobile electronicdevice 150 to perform image processing, data storage, and the like.

Example Thermal Imaging System for a Personal Electronics Device

FIGS. 2A and 2B illustrate an example thermal camera 201 configured tobe coupled to a personal electronics device 203. The thermal camera 201can include components configured to acquire thermal image data and canbe configured to transmit this thermal image data to the personalelectronics device 203 via a connector 202. The personal electronicsdevice 203 can further process the image data from the thermal camera201 to apply filters to the image data, provide thermography functions,display thermal image data, and the like. As described herein withreference to FIG. 1B, the personal electronics device 203 can beconfigured to complement, supplement, augment, and/or complete theimaging capabilities of the thermal camera 201. The personal electronicdevice 203, for example, can be used to control and to provide userinput used by the thermal camera 201. For example, to acquire a thermalimage, an application can run on the personal electronics device 203 anda user interaction with the personal electronics device 203 (e.g., bytouching or otherwise interacting with a touchscreen) can initiate theprocess of acquiring a thermal image with the thermal camera 201. Theacquired image data can be processed by a combination of the thermalcamera 201 and the personal electronics device 203 for display on thescreen of the personal electronics device 203. In some embodiments,interaction with the thermal camera 201 occurs through an applicationrunning on the personal electronics device 203.

To provide greater information on the display of the personalelectronics device 203, for example, a thermal image acquired with thethermal camera 201 can be displayed using a combination of colors and/orgrey scale. To decide which pixels are to be displayed using a colorscale and which pixels are to be displayed using grey scale, criteriacan be analyzed. In some embodiments, the criteria can be determinedbased at least in part on interaction with the personal electronicsdevice 203 (e.g., through user input acquired through interaction with atouchscreen). In some implementations, the criteria can be automaticallydetermined. By displaying at least a portion of an imaged scene using acolor scale and the rest of the imaged scene using a grey scale, thecolor portions of the image can be readily identified and informationcan be easily deduced from such a display. For example, a user viewing ascene that is partially displayed in color can easily identify whichportions of the scene are in color and therefore the user can easilyidentify which portions of the scene meet the defined criteria. This canresult in the user quickly and easily determining which portions of thescene being imaged meet designated temperature criteria, for example.Thus, the thermal camera 201 can be used to provide a quick and easy wayto determine thermal properties of a scene using a display of aconnected personal electronics device 203, wherein at least a portion ofthe scene is configured to be displayed using a first color scale (e.g.,through the use of a color lookup table) and another portion of thescene is configured to be displayed using a second color scale or a greyscale.

In some embodiments, the thermal camera 201 can be of a variety ofcamera or imaging accessory for a personal electronics device 203intended for mass-produced, personal use. It is to be understood,however, that the methods and systems described herein apply to otherconfigurations including a thermal imaging system that is an independentdevice with its own processor and display, as well as intermediateconfigurations (e.g., a thermal imaging system with a detachable orsecondary display).

Example System for Selective Color Display of a Thermal Image

FIG. 3 illustrates a functional block diagram of a thermal imagingsystem 300 configured to convert thermal image data acquired with athermal image sensor 305 to an array of display-formatted pixels,wherein at least two different lookup tables are applied to thedisplay-formatted pixels depending on defined or selected criteria.Thus, in certain implementations, a thermal imaging system can beconfigured to acquire thermal image data corresponding to an array ofpixel intensity values and to display a thermal image wherein a colorlookup table is applied to a first plurality of display-formatted pixelsof the thermal image and a grey-scale lookup table is applied to asecond plurality of display-formatted pixels of the thermal image, thefirst plurality of display-formatted pixels corresponding to pixels inthe array of pixel intensity values whose intensity values satisfy acriteria based at least in part on scene temperature.

The thermal imaging system 300 can include a thermal image sensor 305configured to acquire thermal image data 310. The thermal image sensor305 can be a focal plane array, as described herein. The thermal imagesensor 305 can be configured to thermally image scenes and to produceframes of thermal image data 310 of those scenes for display (and/orcapture). The thermal image data 310 can comprise an array of pixelintensity values, wherein the intensity values correspond totemperatures within a scene being imaged.

The thermal imaging system 300 can include image processing electronics315 configured to adjust the thermal image data 310 based on calibrationvalues, filters, non-uniformity corrections, gain corrections, and thelike. Examples of image processing functions and systems are describedherein in greater detail with reference to FIGS. 1A and 1B. The imageprocessing electronics 315 can output an array of processed pixelintensity values 320. In some embodiments, the thermal image sensor 305includes readout electronics that output 14-bit digital words for eachpixel so that the array of pixel intensity values 320 is an array of14-bit digital words. The image processing electronics 315 can beconfigured to maintain the length of each digital word throughout imageprocessing while adjusting the value of individual pixels.

The thermal imaging system 300 can include a thermography component 325configured to convert between pixel intensity values and temperaturesusing a thermography function. The thermography component 325 isconfigured to correlate measured intensity per processed pixel to scenetemperature through a series of calibration and computational steps.Similarly, the thermography component 325 is configured to correlatetemperature values to measured intensity values. For example, thethermography component 325 can receive a temperature and convert thatinto an equivalent pixel intensity value. This can be useful, asdescribed herein, to convert temperature-based criteria tointensity-based criteria. The intensity-based criteria can then be usedto determine whether individual pixels satisfy the temperature-basedcriteria using intensity-based comparisons. Examples of thermographysystems and methods are described in greater detail in U.S. patent.application Ser. No. 14/838,000, entitled “Thermography for a ThermalImaging Camera,” filed Aug. 27, 2015, the entire contents of which isincorporated by reference herein.

The thermography component 325 can be configured to receive criteriainformation 330 comprising one or more temperature conditions related tothe display of thermal images. In some embodiments, the criteriainformation 330 is received from a user. For example, a user caninteract with a display or other input device to define temperaturecriteria for display. In some embodiments, the criteria information 330is determined automatically by the thermal imaging system 300. Forexample, the thermal imaging system 300 can be configured to determinecriteria information to identify regions of interest in an image (e.g.,by identifying outlier pixels based on a statistical analysis of this orpreceding image frames). As another example, the thermal imaging system300 or other system can be configured to determine criteria informationbased on pre-defined rules. In various implementations, thesepre-defined rules may be defined by a user rather than the userexplicitly defining temperature criteria. In some embodiments, thecriteria information 330 is received from another system. Thetemperature criteria can include one or more temperature thresholdsand/or one or more temperature ranges. Pixels that meet the temperaturecriteria can be formatted for display differently than pixels that donot meet the temperature criteria. Pixels that meet one set oftemperature conditions within the criteria can be formatted for displaydifferently than pixels that meet another set of temperature conditionswithin the criteria.

Examples of temperature criteria can include a minimum temperaturethreshold. In this example, pixel intensity values corresponding totemperatures that exceed the minimum temperature threshold can bedisplayed using a different lookup table applied to thedisplay-formatted pixels than pixels that have intensity valuescorresponding to temperatures that are less than or equal to the minimumtemperature threshold. Similarly, examples of temperature criteria caninclude a maximum temperature threshold. In this example, pixelintensity values corresponding to temperatures that are less than themaximum temperature threshold can be displayed using a different lookuptable applied to the display-formatted pixels than pixels that haveintensity values corresponding to temperatures that are greater than orequal to the maximum temperature threshold. Similarly, examples oftemperature criteria can include a targeted temperature value. In thisexample, pixel intensity values corresponding to temperatures that aresubstantially the same as the targeted temperature value (or within asmall range of values around the targeted temperature value) can bedisplayed using a different lookup table applied to thedisplay-formatted pixels than pixels that have intensity valuescorresponding to temperatures that are different from the targetedtemperature value. In various implementations, a temperature value canbe substantially the same as the targeted temperature value when thetemperature value is within the temperature measurement accuracy of thethermal imaging system 300. Similarly, examples of temperature criteriacan include a targeted temperature range. In this example, pixelintensity values corresponding to temperatures that are within thetargeted temperature range can be displayed using a different lookuptable applied to the display-formatted pixels than pixels that haveintensity values corresponding to temperatures that are outside of thetargeted temperature range. The temperature criteria can also include acombination of these example criteria. In such a configuration, a firstlookup table can be used for pixels meeting a first criteria, a secondlookup table can be used for pixels meeting a second criteria, and soon. Pixels that do not meet any criteria can be displayed using adefault lookup table. In some embodiments, the lookup tables for pixelsthat meet the defined criteria can correspond to color lookup tables andthe default lookup table can correspond to a grey-scale lookup table.

The thermography component 325 can be configured to convert the criteriainformation 330 into intensity-based criteria for analysis of theprocessed pixel intensity values 320. In some embodiments, thethermography component 325 can receive the processed pixel intensityvalues 320 and can generate an array of temperature values 340corresponding to the processed pixel intensity values 320.

The thermal imaging system 300 can include criteria analysis component335 configured to determine which pixels in the processed pixelintensity values 320 meet the one or more conditions in the criteriainformation 330. As described above, one or more conditions can bepresent in the criteria information 330 and the criteria analysiscomponent 335 can be configured to determine which conditions, if any,are met for individual pixels. The criteria analysis component 335 canbe configured to control selection, as described herein, of a lookuptable to apply to individual pixels based on the outcome of thecomparison of individual pixel intensity value to the one or moreconditions.

In some embodiments, the criteria analysis component 335 can beconfigured to receive the array of temperature values 340 and compareindividual temperature values to the criteria information 330 comprisingone or more conditions. In such an embodiment, the criteria information330 may skip the conversion from temperature to intensity as provided bythe thermography component 325. In such an embodiment, the criteriaanalysis component 335 can be configured to control selection of alookup table to apply to individual pixels based on the outcome of thecomparison of individual pixel temperature values to the one or moreconditions.

The thermal imaging system 300 includes the display processing component345 configured to generate an array of display-formatted pixels 350. Thedisplay processing component 345 receives the processed pixelintensities 320 and performs histogram equalization processes on thepixel data. Examples of such processes are described herein withreference to FIG. 1A as well as in U.S. patent application Ser. No.14/292,124, entitled “Data Digitization and Display for an ImagingSystem,” filed May 30, 2014, the entire contents of which isincorporated by reference herein. In some embodiments, the array ofprocessed pixel intensity values 320 can be adjusted using histogramequalization methods, for example, to represent each pixel with a valueappropriate for a display of the thermal imaging system 300. In someembodiments, the thermal image sensor 305 outputs 14-bit digital wordsfor each pixel and the display processing component 345 is configured toadjust individual pixel values so that each is represented using an8-bit digital word. In such embodiments, the display-formatted pixels350 comprise an array of 8-bit digital words.

A lookup table selection component 355 can receive the display-formattedpixels 350 and apply an appropriate lookup table based on the outcome ofthe criteria analysis component 335. In some embodiments, for anindividual pixel, the criteria analysis component 335 can determinewhether the individual pixel meets any of the criteria 330 and canprovide that result to the lookup table selection component 355 so thatit applies a suitable lookup table to the display-formatted pixel fordisplay on the display 365.

The appropriate lookup table can be based at least in part on thecriteria met, if any, by the individual pixel. For example, a pixel thatmet a first condition can be formatted for display by reference to afirst lookup table (LUT 1) 357 a. Similarly, a pixel that met a secondcondition can be formatted for display by reference to a second lookuptable (LUT 2) 357 b. Likewise, a pixel that did not meet any criteriacan be formatted for display by reference to a third lookup table (LUT3) 357 c. Additional lookup tables may be present for selection in thelookup table selection component 355. In some embodiments, one or morelookup tables 357 may not be used when displaying a thermal image. Insome embodiments, at least two different lookup tables are applied todifferent display-formatted pixels.

The lookup tables 357 can be configured to map a color value to adisplay-formatted pixel value. This mapping can be one-to-one. Forexample, where the display-formatted pixels are 8-bit digital words(corresponding to 256 different values per pixel), individual lookuptables 357 can have 256 entries that map an input digital word to acolor value for display on the display 365 of the thermal imaging system300.

The lookup tables 357 can be color lookup tables (e.g., a lookup tablethat maps display-formatted pixel values to one or more differentcolors) or a grey-scale lookup table (e.g., a lookup table that mapsdisplay-formatted pixel values to a grey-scale value). For example, acolor lookup table can map input pixel values to color values that rangefrom black to white with yellow and red colors assigned to intermediatevalues. A grey-scale lookup table can be configured to map input pixelvalues to different grey levels from black to white (or from white toblack).

Using an individual pixel to illustrate the process, the pixel from thethermal image sensor 305 can be processed by the image processingelectronics 315 to adjust its value. The pixel can then be analyzed todetermine whether it meets any defined criteria 330. The pixel is thenprocessed for display by the display processing component 345 whichreceives the pixel and adjusts its value using histogram equalizationmethods. An appropriate lookup table is applied to the display-formattedpixel value based on the result of the criteria analysis. For example,if the processed pixel intensity value meets the criteria 330, the colorlookup table 357 a can be applied to map the display-formatted pixelvalue to a color (e.g., orange).

Different color regimes in different lookup tables may be used in thelookup table selection component 355. For example, a red and yellowcolor regime may be used for criteria corresponding to highertemperatures and a blue and purple regime may be used for criteriacorresponding to lower temperatures. Such a configuration may beimplemented where there is a single criteria (e.g., a temperaturethreshold value) and the lookup table can be automatically selecteddepending on the single criteria. For example, if the temperaturethreshold value is below a configured temperature than a color lookuptable using “cooler” colors (e.g., blues, purples, greens, etc.) can beused and a color lookup table using “warmer” colors (e.g., yellows,oranges, reds, etc.) can be used where the temperature threshold valueis above the configured temperature. Such configurations may beimplemented where there are multiple criteria and the lookup tables canbe automatically assigned to different criteria depending on thetemperatures or temperature ranges associated with the individualcriteria. For example, where a first criteria corresponds totemperatures that are lower than temperatures associated with a secondcriteria, a color lookup table using “cooler” colors (e.g., blues,purples, greens, etc.) can be associated with the first criteria and acolor lookup table using “warmer” colors (e.g., yellows, oranges, reds,etc.) can be used with the second criteria. In certain implementations,a user can assign individual criteria to individual lookup tables. Theuser may also be able to define color configurations for use.

In some embodiments, the display 365 receives temperature values 340 fordisplay. The display 365 can be configured to display at least onetemperature from the temperature values 340 overlaid on the thermalimage. In certain embodiments, the at least one temperature is displayedin a position at or near the part of the imaged scene that is at thattemperature.

FIGS. 4A and 4B respectively illustrate example displayed thermal images400 a, 400 b wherein the criteria correspond to a minimum temperaturevalue. For example, FIG. 4A illustrates a displayed thermal image 400 awherein the criteria corresponds to a minimum temperature of 84° F. suchthat pixels corresponding to portions of the scene with temperaturesgreater than or equal to 84° F. are displayed using a color lookup table(colorized pixels 402 a). FIG. 4B illustrates a displayed thermal image400 b wherein the criteria corresponds to a minimum temperature of 79°F. such that pixels corresponding to portions of the scene withtemperatures greater than or equal to 79° F. are displayed using a colorlookup table (colorized pixels 402 b). For each displayed thermal image400 a, 400 b, portions of the scene with a temperature below therespective temperature thresholds are displayed using a grey-scalelookup table (grey scale pixels 404 a, 404 b). Note that temperature ofthe person's hair in the scene falls between the two thresholds. For thedisplayed thermal images 400 a, 400 b, the display is on a touchscreenof a personal electronics device and the minimum temperature value 405a, 405 b is set by a user through interaction with a touch-activatedslider 410 a, 410 b.

FIGS. 5A and 5B respectively illustrate example displayed thermal images500 a. 500 b wherein the criteria correspond to a targeted temperaturevalue. For example, FIG. 5A illustrates a displayed thermal image 500 awherein the criteria corresponds to a targeted temperature of 84° F.such that pixels within the accuracy of the thermal imaging system(e.g., about ±1° F.) of the targeted temperature value are displayedusing a color lookup table (colorized pixels 502 a). FIG. 4B illustratesa displayed thermal image 400 b wherein the criteria corresponds to atargeted temperature of 79° F. such that pixels within the accuracy ofthe thermal imaging system (e.g., about ±1° F.) of the targetedtemperature value are displayed using a color lookup table (colorizedpixels 502 b). For each displayed thermal image 500 a, 500 b, pixelscorresponding to portions of the scene with a temperature different fromthe respective targeted temperature values are displayed using agrey-scale lookup table (grey scale pixels 504 a, 504 b). It should benoted that the accuracy of the thermal imaging system may differ fordifferent implementations so that the range of temperatures consideredto be equal to or substantially the same as the targeted temperaturevalue can differ for these different implementations. For example,thermal imaging systems that are less accurate in determiningtemperature can be configured to determine that a larger range oftemperatures are equal to the targeted temperature value than moreaccurate systems.

Example Method of Selective Color Display of a Thermal Image

FIG. 6 illustrates an example method 600 of selective color display of athermal image. The method 600 can be implemented using one or morehardware components in a thermal imaging system or image processingsystem. For ease of description, the method 600 will be described asbeing performed by the imaging system 100 described herein withreference to FIGS. 1A and 1B. However, one or more of the steps of themethod 600 can be performed by any module, such as the displayprocessing module 114, or combination of modules in the imaging system100. Similarly, any individual step can be performed by a combination ofmodules in the imaging system 100. Likewise, the steps of the method canbe performed by the thermal imaging system 300 described herein withreference to FIG. 3.

In block 605, the imaging system receives thermal image data acquiredwith a thermal image sensor. The imaging system can include the thermalimage sensor or the imaging system can receive the thermal image datafrom another system (e.g., a thermal imaging camera or a data storagesystem). The thermal image data can include an array of pixel intensityvalues, the intensities of individual pixels corresponding to anintensity of infrared radiation detected by the pixel.

In block 610, the imaging system converts temperature-based criteria tointensity-based criteria. In some embodiments, the criteria can bereceived through user interaction with the imaging system. This canallow a user to determine desired temperature criteria for displayformatting. In certain embodiments, the criteria can be provided fromanother system. This can allow another system to automatically determinetemperature criteria and send the criteria with acquired thermal imagesto the imaging system. In various embodiments, the criteria can beautomatically determined by the imaging system. This can allow theimaging system to analyze the thermal image data to automaticallydetermine temperature criteria for display formatting. The temperaturecriteria can be converted to intensity-based criteria using athermography function. The thermography function, for example, can beconfigured to convert from a temperature to an equivalent pixelintensity value.

In block 615, the imaging system compares pixel values to thetemperature criteria. In certain implementations, the pixel values areintensity values (e.g., values corresponding to an intensity of infraredlight at the pixel) that have been processed by the imaging system(e.g., by performing calibration procedures, applying non-uniformitycorrections, applying filters, etc.). In such implementations, thecomparison is accomplished by comparing individual pixel intensityvalues to the intensity-based criteria.

In various implementations, the pixel values are temperature values thathave been determined using the thermography function. The temperaturevalues can be determined after the pixel intensity values have beenprocessed using any of the signal processing methods or modulesdescribed herein. In such implementations, the temperature criteria canremain as temperature-based values and the comparison can be between theconverted temperature values of the pixels and the temperature-basedvalues of the temperature criteria.

In block 620, the imaging system applies a lookup table to individualdisplay-formatted pixels, the applied lookup table depending at least inpart on the comparison of the pixel value to the temperature criteria.The imaging system can include a plurality of lookup tables. For pixelssatisfying the temperature criteria, a first lookup table can be appliedto the display-formatted pixels. In some embodiments, the first lookuptable can be a color lookup table. For pixels that do not satisfy thetemperature criteria, a second lookup table can be applied to thedisplay-formatted pixels. In some embodiments, the second lookup tablecan be a grey-scale lookup table or a second color lookup table (e.g., acolor table with colors that are different and/or distinct from thecolors in the first lookup table). In some embodiments, the temperaturecriteria can include a plurality of conditions that can be independentlysatisfied. In such embodiments, individual lookup tables can beassociated with particular conditions in the temperature criteria.Satisfaction of the particular conditions can mean that the imagingsystem applies the associated lookup table to the display-formattedpixels. In this way, multiple color lookup tables can be usedsimultaneously along with a grey-scale lookup table. This canadvantageously provide a way for a user to readily identify pixels thatsatisfy individual conditions among a plurality of defined conditions.

The lookup tables can be configured to map display-formatted pixelvalues to color values. As described herein, the display-formatted pixelvalue can be a digital word. In certain implementations, the lookuptables can be configured to map a digital word (or array of digitalwords) to output color values.

In block 625, the imaging system displays a thermal image comprising thedisplay-formatted pixels with the appropriate lookup tables applied. Insome embodiments, at least a portion of the pixels are displayed using agrey-scale and another portion of the pixels are displayed in color. Inthis way, a user can relatively quickly and easily determine whichpixels meet the defined criteria.

The embodiments described herein are exemplary. Modifications,rearrangements, substitute processes, etc. may be made to theseembodiments and still be encompassed within the teachings set forthherein. One or more of the steps, processes, or methods described hereinmay be carried out by one or more processing and/or digital devices,suitably programmed.

Depending on the embodiment, certain acts, events, or functions of anyof the algorithms described herein can be performed in a differentsequence, can be added, merged, or left out altogether (e.g., not alldescribed acts or events are necessary for the practice of thealgorithm). Moreover, in certain embodiments, acts or events can beperformed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors or processor cores or onother parallel architectures, rather than sequentially.

The various illustrative logical blocks, modules, and algorithm stepsdescribed in connection with the embodiments disclosed herein can beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, and stepshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. The described functionality can be implemented invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a processor configured with specificinstructions, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A processor can be amicroprocessor, but in the alternative, the processor can be acontroller, microcontroller, or state machine, combinations of the same,or the like. A processor can also be implemented as a combination ofcomputing devices, e.g., a combination of a DSP and a microprocessor, aplurality of microprocessors, one or more microprocessors in conjunctionwith a DSP core, or any other such configuration. For example, the LUTdescribed herein may be implemented using a discrete memory chip, aportion of memory in a microprocessor, flash, EPROM, or other types ofmemory.

The elements of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module executed by a processor, or in acombination of the two. A software module can reside in RAM memory,flash memory, ROM memory, EPROM memory, EEPROM memory, registers, harddisk, a removable disk, a CD-ROM, or any other form of computer-readablestorage medium known in the art. An exemplary storage medium can becoupled to the processor such that the processor can read informationfrom, and write information to, the storage medium. In the alternative,the storage medium can be integral to the processor. The processor andthe storage medium can reside in an ASIC. A software module can comprisecomputer-executable instructions which cause a hardware processor toexecute the computer-executable instructions.

Conditional language used herein, such as, among others, “can,” “might,”“may,” “e.g.,” and the like, unless specifically stated otherwise, orotherwise understood within the context as used, is generally intendedto convey that certain embodiments include, while other embodiments donot include, certain features, elements and/or states. Thus, suchconditional language is not generally intended to imply that features,elements and/or states are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without author input or prompting, whether thesefeatures, elements and/or states are included or are to be performed inany particular embodiment. The terms “comprising,” “including,”“having,” “involving,” and the like are synonymous and are usedinclusively, in an open-ended fashion, and do not exclude additionalelements, features, acts, operations, and so forth. Also, the term “or”is used in its inclusive sense (and not in its exclusive sense) so thatwhen used, for example, to connect a list of elements, the term “or”means one, some, or all of the elements in the list.

Disjunctive language such as the phrase “at least one of X, Y or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y or Z, or any combination thereof (e.g., X, Y and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y or at least one of Z to each be present.

The terms “about” or “approximate” and the like are synonymous and areused to indicate that the value modified by the term has an understoodrange associated with it, where the range can be ±20%, ±15%, ±10%, ±5%,or ±1%. The term “substantially” is used to indicate that a result(e.g., measurement value) is close to a targeted value, where close canmean, for example, the result is within 80% of the value, within 90% ofthe value, within 95% of the value, or within 99% of the value.

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

While the above detailed description has shown, described, and pointedout novel features as applied to illustrative embodiments, it will beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As will berecognized, certain embodiments described herein can be embodied withina form that does not provide all of the features and benefits set forthherein, as some features can be used or practiced separately fromothers. All changes which come within the meaning and range ofequivalency of the claims are to be embraced within their scope.

What is claimed is:
 1. A method for displaying a thermal image using athermal imaging system comprising an array of photodetectors configuredto acquire thermal image data, the method comprising: acquiring thermalimage data with the array of photodetectors, the thermal image datacomprising an array of pixel intensity values; receiving temperaturecriteria; determining a scene temperature for individual pixel intensityvalues using a thermography function; comparing individual pixels to thetemperature criteria; formatting for display individual pixel values;applying a first lookup table to an individual display-formatted pixelif the pixel value satisfies the temperature criteria and applying asecond lookup table to the individual display-formatted pixel if thepixel value does not satisfy the temperature criteria; and displaying athermal image on a display of the thermal imaging system, the thermalimage comprising the display-formatted individual pixel values with theapplied lookup tables.
 2. The method of claim 1 further comprisingconverting the temperature criteria to intensity-based criteria using athermography function.
 3. The method of claim 2, wherein comparingindividual pixels to the temperature criteria comprises comparingindividual pixel intensity values to the intensity-based criteria. 4.The method of claim 1, wherein comparing individual pixels to thetemperature criteria comprises comparing individual pixel temperaturevalues to the temperature criteria.
 5. The method of claim 1, whereinthe first lookup table is configured to output a color value and thesecond lookup table is configured to output a grey-scale value, andwherein at least a portion of the thermal image is displayed using colorand another portion of the thermal image is displayed using a greyscale.
 6. The method of claim 1, wherein the temperature criteriacomprises at least one of: a minimum temperature threshold that issatisfied if an individual pixel intensity value corresponds to atemperature that is greater than or equal to the minimum temperaturethreshold; a maximum temperature threshold that is satisfied if anindividual pixel intensity value corresponds to a temperature that isless than or equal to the maximum temperature threshold; a targetedtemperature value that is satisfied if an individual pixel intensityvalue corresponds to a temperature that is substantially the same as thetargeted temperature value; and a targeted temperature range that issatisfied if an individual pixel intensity value corresponds to atemperature that is within the targeted temperature range.
 7. The methodof claim 1, wherein receiving the temperature criteria comprisesdetermining the temperature criteria based on user interaction with thethermal imaging system.
 8. The method of claim 1, wherein thetemperature criteria includes a plurality of conditions.
 9. The methodof claim 8 further comprising applying a first color lookup table if theindividual pixel value satisfies a first condition of the temperaturecriteria, applying a second color lookup table if the individual pixelvalue satisfies a second condition of the temperature criteria, andapplying a grey-scale lookup table if the individual pixel value doesnot satisfy any of the plurality of conditions.
 10. The method of claim1 further comprising displaying at least one scene temperature overlaidon the thermal image.
 11. The method of claim 1 further comprisingperforming signal processing on the thermal image data, the signalprocessing including performing a histogram equalization method on thepixel intensity values prior to applying the first or second lookuptables.
 12. The method of claim 1 further comprising displaying a userinterface with which a user may interact to set the temperaturecriteria.
 13. A thermal imaging system comprising: an imaging arraycomprising an infrared focal plane array, the infrared focal plane arrayconfigured to generate signals corresponding to levels of infrared lightincident on the infrared focal plane array; a detector circuitcomprising readout electronics that receive the generated signals andoutput an array of pixel intensity values; a system controllerconfigured to: receive temperature criteria; determine a scenetemperature for individual pixel intensity values using a thermographyfunction; compare individual pixels to the temperature criteria; formatfor display individual pixel intensity values; and apply a first lookuptable to an individual display-formatted pixel if the individual pixelintensity value satisfies the intensity-based criteria and apply asecond lookup table to the individual display-formatted pixel if theindividual pixel intensity value does not satisfy the intensity-basedcriteria; and a display configured to display a thermal image comprisingan array of the display-formatted pixels with the applied lookup tables.14. The thermal imaging system of claim 13, wherein the systemcontroller is further configured to convert the temperature criteria tointensity-based criteria using the thermography function.
 15. Thethermal imaging system of claim 14, wherein the system controller isfurther configured to compare individual pixels to the temperaturecriteria by comparing individual pixel intensity values to theintensity-based criteria.
 16. The thermal imaging system of claim 13,wherein the first lookup table comprises a color lookup table and thesecond lookup table comprises a grey-scale lookup table such that thethermal image comprises at least a first plurality of pixels displayedusing color and a second plurality of pixels displayed using grey scale.17. The thermal imaging system of claim 13, wherein the displaycomprises a touchscreen interface and the temperature criteria isreceived through user interaction with the touchscreen interface. 18.The thermal imaging system of claim 13 further comprising: a thermalcamera comprising a thermal camera housing and a connector wherein theimaging array and the detector circuit are within the thermal camerahousing; and a personal electronics device comprising the systemcontroller and the display, wherein the connector of the thermal camerais configured to mechanically and electrically couple to the personalelectronics device to transmit the thermal image data from the thermalcamera to the personal electronics device.
 19. The thermal imagingsystem of claim 18, wherein, in use, control of the thermal camera isprovided through interaction with the personal electronics device. 20.The thermal imaging system of claim 13, wherein the temperature criteriacomprises at least one of: a minimum temperature threshold that issatisfied if an individual pixel intensity value corresponds to atemperature that is greater than or equal to the minimum temperaturethreshold; a maximum temperature threshold that is satisfied if anindividual pixel intensity value corresponds to a temperature that isless than or equal to the maximum temperature threshold; a targetedtemperature value that is satisfied if an individual pixel intensityvalue corresponds to a temperature that is substantially the same as thetargeted temperature value; and a targeted temperature range that issatisfied if an individual pixel intensity value corresponds to atemperature that is within the targeted temperature range.